Robotic work cell and network

ABSTRACT

A robotic network includes multiple work cells that communicate with a cloud server using a network bus (e.g., the Internet). Each work cell includes an interface computer and a robotic system including a robot mechanism and a control circuit. Each robot mechanism includes an end effector/gripper having integral multimodal sensor arrays that measure physical parameter values (sensor data) during interactions between the end effector/gripper and target objects. The cloud server collects and correlates sensor data from all of the work cells to facilitate efficient diagnosis of problematic robotic operations (e.g., accidents/failures), and then automatically updates each work cell with improved operating system versions or AI models (e.g., including indicator parameter value sets and associated secondary robot control signals that may be used by each robot system to detect potential imminent robot accidents/failures during subsequent robot operations.

RELATED APPLICATION

This application claims priority from U.S. Provisional PatentApplication No. 62/826,865, entitled “Robotik Smart Networked Work CellArchitecture”, filed on Mar. 29, 2019, and from U.S. Provisional PatentApplication No. 62/826,883, entitled “Robotic Gripper with IntegratedMulti-Modal Tactile Sensor Arrays”, filed on Mar. 29, 2019.

FIELD OF THE INVENTION

This invention relates generally to robotic systems and moreparticularly to robotic systems having tactile sensors andre-programmable robot mechanism controllers.

BACKGROUND OF THE INVENTION

Most modern robotic systems integrate mechanical, electrical/electronicand computer science technologies to provide autonomously controlledmechanisms capable of performing a variety of programmed roboticoperations (tasks). For example, articulated robots are a class ofindustrial robotic systems in which a control circuit convertsuser-provided software-based instructions into motor control signalsthat control a robot arm mechanism and attached end effector (i.e., anend-of-arm-tooling device, such as a robotic gripper that performsfunctions similar to those performed by a human hand) to performrepetitive tasks, such as grasping target objects while the robot armsmoves the target objects from one location to another location. Toperform such programmed operations, the software-based instructionsprovided to most articulated robots must specify three-dimensional (3D)coordinates of the starting location at which the target objects arelocated for pick-up, a designated 3D travel path through which thetarget objects may be moved without interference, and 3D coordinatesdefining the terminal location (e.g., a receptacle or support surface)at which the target objects are to be placed. When suitablesoftware-based instructions are provided for performing a specificrobotic operation, the system controller generates a correspondingseries of motor control signals that cause the robot arm mechanism tomove the end effector to the initial/starting location coordinates, thencause the gripper to close on (grasp) the target object, then cause therobot arm mechanism to lift/move the target object to the terminallocation coordinates along the designated travel path, and then causethe gripper to release the target object.

A significant limitation of conventional robotic systems is their lackof human-hand-type sensory input that would allow adjustment to a widerange of common operational irregularities and/or to perform advancerobotic operations. That is, unlike the sensor systems found in humanhands that provide multimodal sensory feedback (i.e., mechanoreceptorssensing both pressure and vibration, and thermoreceptors sensingtemperature), the robotic grippers utilized in most conventional roboticsystems utilize no sensing architecture, and those that do utilizesingle-modality sensing architectures (e.g., pressure sensing only) orvisual input via camera system. The multimodal sensing architecturefound on human hands provides fine-grained cues about contact forces,textures, local shape around contact points, and deformability, all ofwhich are critical for evaluating an ongoing grasping operation, and totrigger force correction measures in case of instability. In contrast,conventional robotic systems that utilize grippers having no tactilesensing architecture or visual input rely entirely on pre-programmedcommands, whereby these systems typically fail to adjust for minorpositional variations to unanticipated environmental variations. Forexample, even a small positional displacement of a target object awayfrom its program-designated starting location coordinates may preventsuccessfully grasping by the gripper, and in some cases may result indamage to the target object and/or gripper (e.g., due to off-centercontact between the gripper and the target object during the graspingoperation). While grippers having single-modality sensing architecturesand/or camera systems provide some feedback information to the hostrobotic system's control circuit, thereby allowing the control circuitto modify user-provided program instructions in order to correct for asmall number of specific operating environment irregularities (e.g.,minor target object positional variations relative to expectedprogrammed coordinates), they are incapable of addressing a wide rangeof operating environment irregularities and/or perform more advancedrobotic operations. For example, although single-modality pressuresensors may provide sufficient data to verify that a predeterminedgripping force is being applied onto a target object, suchsingle-modality pressure sensors are unable to recognize when the targetobject may be slipping from the gripper's grasp, and therefore areunable to avoid the resulting accident damage to the target object ifdropped.

Although it would be possible to enhance the capabilities ofconventional arm-type robotic systems by way of adding additionalsensors, simply adding sensors to existing gripper structures would beproblematic. First, the addition of gripper-mounted sensors usingconventional methods would require signal lines (one for each sensor)that would extend along the arm mechanism from each sensor to the systemcontroller; this requirement would both limit the number of sensors thatcould be accommodated, and could also present weight-related issues whenlarge numbers of signal lines are required. Moreover, the roboticsystem's controller would require substantial modification and/orreprogramming to facilitate processing of data received from a largenumber of sensors.

Another limitation of conventional robotic systems is that there is noefficient way to detect, correct and update the operating (software)programs of multiple similar robotic systems to address an accident orlearning event that is experienced by one of the robotic systems (i.e.,so that the remaining robotic systems avoid the same accident and/oradapt the learning event to achieve better performance). That is,conventional robotic systems fail to include sensors capable ofdetecting operating parameters related to the occurrence of an accident(e.g., applied gripping force when an object slips from a gripper'sgrasp), and fail to include a mechanism that can automatically apply acorrective measure capable of preventing a repeat of the accident (e.g.,increase the applied gripping pressure when the object begins to slip).Moreover, the operating system/program of a given conventional roboticsystem is typically entirely independent from (i.e., does notcommunicate with) the operating systems of other similarly configuredrobotic systems—this independent operating system paradigm preventsprogram updates (modifications) that may be developed for one roboticsystem from being efficiently shared with similar robotic systems. Byway of example, assume the identical operating programs provided withmultiple similar robotic systems are configured to calculate an appliedgripping force of ten kilopascals (kPa) for purposes of grasping/movinga hypothetical standardized object, and that all of the robotic systemsare utilized to grasp/move the standardized object in a similar manner.Assume also that one of these robotic systems experiences intermittentaccidents involving dropped objects, and that the actual cause is anintermittent temporary drop in friction between the gripper's fingersand a grasped object that occurs when the robot encounters certainenvironmental conditions (e.g., temperatures below 10° C.). It would bevery difficult to determine the cause of the intermittent accidents whenthey occur in conventional robotic systems that do not include anysensors. Further, conventional robotic systems that includesingle-modality pressure sensors may detect that the applied grippingpressure was maintained at ten kPa during each intermittent accident,but this information would be insufficient to correlate the accidentswith cold temperatures, and would be unable to provide a softwarecorrection (e.g., increase grip pressure on cold days) that couldprevent repeats of the accident in the future. Further, even if anoperator properly deduces that the intermittent accidents are related tolow operating temperatures and modifies the robotic system's operatingprogram to apply twelve kPa of gripping force on cold days, thecorrective measure would only be applied to the operator's roboticsystem. That is, although the operating program of the operator'srobotic system may be successfully modified to the intermittentcold-object-drop accident, conventional robotic systems do not provide amechanism for this operating program modification to be efficientlyconveyed to the other similar robotic systems that might, in exchange,provide corrective actions that address other types ofaccidents/failures. As a result, the overall average performance ofmultiple similarly configured conventional robotic systems is destinedto remain sub-optimal, thereby wasting valuable energy and resourcesthat could be spend on more beneficial pursuits.

What is needed is a cooperative arrangement in which the operatingperformance of multiple similarly configured robotic systems isefficiently improved. For example, what is needed is an arrangementcapable of diagnosing the causes of robotic operation accidents/failuresthat are commonly experienced by the multiple robotic systems, andcapable of efficiently modifying all of the robotic systems to includesubsequently developed corrective actions that prevent or mitigatereoccurrences of the accidents/failures. Preferably, such an arrangementwould be configured to prevent proprietary information of each roboticsystem's user/owner (e.g., command sequences developed to execute aspecific robot operation) from being undesirably disclosed to the otherrobotic system users/owners.

SUMMARY OF THE INVENTION

The present invention is directed to a robotic network (and associatednetwork operating method) that utilizes enhanced multimodal sensorconfigurations disposed on multiple robotic systems to measure variousphysical parameters during robot operations involving interactionsbetween each robotic system's end effector and target objects, andutilizes a cloud server to receive the enhanced multimodal sensor datagenerated by the robotic systems, which facilitates the efficientdetection of potential non-optimal operating system processes and thedevelopment of operating system software updates that address (e.g.,improve or optimize) the non-optimal processes, and to automaticallydistribute subsequently developed updated operating system software(e.g., revisions to artificial intelligence (AI) or machine learningmodels) to every robotic system of the network. The robotic networkfacilitates this cooperative interaction by way of including eachindividual robotic system in an associated work cell that also includesan interface computer, where each work cell's interface computerfacilitates communications between a control circuit of its associatedrobotic system and the cloud server over a network bus (e.g., theInternet or a local area network). In a practical embodiment, theinterface computer of each work cell is also configured to allow auser/owner to enter user-designated instructions in order to control theassociated robot system to perform a desired specific robot operation.In a presently preferred embodiment, the robot mechanisms of themultiple robotic system have a similar configuration, where all of therobotic systems generally include a robotic arm that is fixed at one endand has a gripper (or other end effector) connected to its distal (free)end, and where operations performed by the arm and gripper arecontrolled by signals transmitted from a control circuit to arm (first)actuators and gripper (second) actuators disposed on the robot arm andgripper, respectively. To facilitate the collection of enhancedmultimodal sensor data each robotic system includes one or moremultimodal sensor arrays including different types of sensors thatrespectively measure associated physical parameter values (e.g., appliedgripper pressure, target object temperature and other forms of tactileinformation). The control circuit of each associated robotic systemincludes (i.e., is configured to operate in accordance with) anoperating system (i.e., a software program executed by one or moreprocessors, which may include an AI or machine learning model) that ituses during normal operations to convert the user-designatedinstructions into corresponding primary robot control signals that causethe various actuators of the robot mechanism to perform the desiredspecific robot operation. By providing each robotic system of therobotic network with multimodal sensor arrays configured to generatemeasured physical parameter values that provide rich sensory tactiledata describing the robotic system's operating environment, the presentinvention facilitates the accurate diagnosis of non-optimal robotoperations (e.g., by way of recording various physical parameter valuesthat may be utilized to accurately identify the cause of roboticoperation accidents/failures). Moreover, by configuring robotic networksuch that the measured physical parameter values from multiple similarlyconfigured robotic systems is collected (combined together) by the cloudserver, the present invention achieves highly efficient diagnosis ofnon-optimal robot operations by way of enabling comparisons betweensimilar measured physical parameter values received from widelydispersed robot systems. Further, by configuring each robotic systemsuch that operating system updates can be efficiently transferred to thecontrol circuit of each robotic system, the present inventionfacilitates an improvement in overall robot operation performance thatis shared by all of the network's work cells. Finally, by configuringthe robotic network such that the measured physical parameter values areonly received by the cloud server, the present invention facilitateseasily implemented protocols to protect the proprietary information ofdifferent work cell owners/users by, for example, configuring the sharedenhanced multimodal sensor data to omit any proprietary informationand/or by imposing a non-disclosure condition on the cloud serveroperator. In some embodiments the various different deployed work cellscan have different configurations or identical configurations dependingon the user preferences. Taken together, the entire network operates asa collective learning system whose communications take place either inreal-time or through delayed synchronizations.

According to a practical exemplary embodiment, the robotic network isconfigured to improve the overall robot operation performance ofmultiple work cells by way of collecting measured physical parametervalues of a type suitable for diagnosing the causes of robot operationaccidents/failures (e.g., the cause of the intermittent cold-object-dropaccident example introduced in the background section), and by way ofdistributing subsequently developed accident/failure avoidanceoperations to all of the work cells. Providing each robotic system witha multimodal sensor array of the type described above allows eachrobotic system to detect/measure a wide range of environmentalconditions that occur immediately before a given accident/failure,thereby facilitating a detailed diagnosis of the givenaccident/failure's cause(s). In some embodiments the measured physicalparameter values are transmitted and recorded in “raw” form by the cloudserver, and in other embodiments the “raw” values are converted orprocessed into condensed tactile information to facilitate efficienttransmission—in either case, the measured physical parameter valuesprovide a detailed record of the interactions between the end effectorand target objects during multiple robot operations that resulted in theaccident/failure. In one embodiment the accident/failure diagnosisincludes identifying both (i) reoccurrence indicator parameter values(i.e., specific physical parameter values (or ranges thereof) that, whendetected during robot operations, indicate the potential imminentreoccurrence of the given accident/failure), and (ii) at least oneaccident/failure avoidance/mitigation operation (e.g., one or moresecondary robot control signals that, when transmitted to and executedby one or more of the robot mechanism's actuators, causes the robotmechanism to operate in a way that either avoids/prevents reoccurrenceof the accident/failure, or mitigates damage caused by theaccident/failure reoccurrence). Upon completion of the diagnosis processthe cloud server automatically transmits the accident/failure avoidanceoperation to the control circuit of each work cell (i.e., by way of thenetwork bus and each work cell's interface computer) such that all ofthe work cells receive a copy of the accident/failure avoidanceoperation. In one embodiment the accident/failure avoidance operation istransmitted in the form of an updated operating system that replaces ormodifies the existing (initial) operating system included in the controlcircuit, whereby subsequent robot operations are performed in accordancewith the updated operating system. For example, in accordance with theupdated operating system, the control circuit of each work cell may (a)store multiple sets of indicator parameter values in a way thatfacilitates comparison with measured physical parameter values generatedby the multimodal sensor array(s), and (b) during subsequently executionof the owner/user-provided specific robot operation (i.e., transmittingthe primary robot control signals to the robot mechanism actuators),periodically compare currently measured physical parameter values withthe stored sets of indicator parameter values, and execute anaccident/failure avoidance/mitigation operation only when the currentlymeasured physical parameter values match an associated stored set ofindicator parameter values. That is, while the currently measuredphysical parameter values indicate an expected normal operating state(i.e., do not match any of the sets of stored indicator parametervalues), the modified/updated operating system causes the controlcircuit to actuate robot mechanism in the manner produced by theprevious (non-updated) version of the operating system (i.e., assumingthe previous version did not include the accident/failureavoidance/mitigation operation). However, because the modified/updatedoperating system causes the control circuit to compare currentlymeasured physical parameter values with the stored set of indicatorparameter values, if at some point during robot operations a matchoccurs, then the modified/updated operating system causes the controlcircuit to transmit corresponding secondary robot control signals (i.e.,either in combination with the primary robot control signals, or by wayof transmitting corresponding secondary robot control signals whileinterrupting and/or overriding the primary robot control signals),thereby causing the robot mechanism to terminate or modify the primaryrobotic operation in a way defined by the secondary robot controlsignals that prevents or mitigates the corresponding robotaccident/failure. By configuring each work cell of the robotic networksuch that the operating system of each control circuit can beautomatically updated by the cloud server, the present inventionfacilitates an improvement in overall robot operation performance thatis shared by all of the network's work cells by way of implementing newor revised corrective actions that prevent/mitigate the reoccurrence ofcommonly occurring accidents/failures.

In an exemplary embodiment, the proposed methodology is applied to theintermittent cold-object-drop accident example, introduced above. Inthis exemplary embodiment, the proposed methodology includes correlatingmeasured temperature and pressure (physical parameter) values generatedby one or more multimodal sensor arrays prior to or during the previousoccurrence of one or more cold-object-drop accidents in order toidentify an underlying cause (e.g., objects are dropped when theirtemperature is below 10° C.). Once the accident's cause is identified, asuitable corrective action may be devised that, when performed by eachwork cell, prevents reoccurrence of the cold-object-drop accident. Forexample, experiments may be conducted at various temperatures and grippressures to determine that reoccurrences of cold-object-drop accidentsare reliably prevented by temporarily increasing the applied gripperpressure (e.g., from the normal ten kPa to twelve kPa) whenever acurrently grasped target object's temperature is at or below 10° C.Based on this diagnosis, a corresponding operating system update is thengenerated that includes, for example, indicator parameter valuessuitable for detecting the potential imminent reoccurrence of thecold-object-drop accident (e.g., the indicator parameter values mayinclude a check for currently measured object temperature values equalto or below 10° C.), and also includes one or more correspondingsecondary robot control signals that prevent or mitigate the accident(e.g., a command to increase gripper pressure to twelve kPa whencurrently measured physical parameter values match the stored indicatorparameter value). After being transmitted to the work cells by the cloudserver, this operating system update modifies the performance of thecontrol circuit of each work cell during subsequent robot operations,for example, by causing the control circuit to store the indicatorparameter value (i.e., object temperature greater than or equal to tendegrees Celsius), to compare the stored value with currently measuredtemperature values generated by temperature sensor(s) of the multimodalsensor array, and to increase the gripper pressure to twelve kPa (e.g.,by transmitting the secondary robot control signals to the gripperactuator) whenever a currently measured temperature value is at or below10° C., thereby preventing reoccurrence of cold-object-drop accidents.In other embodiments the indicator parameter values may include any ofthe other measured physical parameter values or a combination thereof,and the secondary robot control signals may be used to perform anysuitable accident preventing operation performed by the robot arm orgripper or a combination thereof, and may be used to perform anoperation designed to mitigate damage caused by an associated accident(i.e., when a suitable accident prevention operation cannot beidentified, the secondary robot control signals may be selected, forexample, to perform an arm/gripper action that functions to minimizedamage to a dropped object, or to minimize damage to the robot mechanismthat may be incurred by a reoccurrence of an associatedaccident/failure).

According to a practical embodiment, the robotic system of each workcell includes an arm-type robot mechanism (robot arm) having a fixedbase portion at one end that is fixedly secured to a base structure(e.g., a table), and a gripper-type end effector (robotic gripper)connected to the opposing distal (free) end portion of the arm-typerobot mechanism. In a presently preferred embodiment, the controlcircuit includes two controller/processor circuits: a robot armcontroller and a gripper controller. The robot arm controller ismounted, for example, on the base/table below the arm-type robotmechanism, and is configured to control the operation of the robot armby way of transmitting arm control signals on wires/conductors extendingalong the robot arm to arm actuators operably disposed on the robot arm(e.g., at connection points between contiguous arm structures). Thegripper controller is operably disposed on (e.g., integrated into) therobotic gripper and configured both to receive and process measuredphysical parameter values generated by the multimodal sensor arraysdisposed on the gripper, and to generate local finger control signalsthat control at least some of the operations (e.g., grasping andreleasing objects and other gripper operations) performed by the roboticgripper. In an exemplary embodiment, the robot arm controller generatesprimary robot control signals during normal robotic operations, wherethe primary robot control signals include both the arm control signalstransmitted to the arm actuators and the system gripper control signalstransmitted to the finger actuators, and the gripper controllergenerates secondary robot control signals only under abnormal operatingconditions (e.g., when a potentially dangerous condition is detected),where the secondary robot control signals include finger control signalsthat control the finger actuators in a predefined way to avoid anundesirable outcome. This arrangement facilitates highly coordinatedarm/gripper operations (i.e., by utilizing the robot arm controller togenerate both the arm control signals and the system gripper controlsignals under normal operating conditions). In addition, by configuringthe robotic system's control circuit to include separate arm and grippercontrollers such that gripper finger operations are directly controlledby the gripper controller when a potential imminent accident/failure isdetected (i.e., by way of transmitting the finger control signalsdirectly to the finger actuators), the present invention increases thesystem's ability to prevent or mitigate the potential accident/failureby substantially reducing latency (i.e., increases the robotic system'sability to quickly respond to potentially harmful or dangerous operatingconditions) without requiring the owner/user of each work cell to takeany overt action.

In one embodiment, each gripper controller is configured to generate thefinger control signals and the tactile information by processing themeasured physical parameter values (sensor data) received from one ormore multimodal sensor arrays disposed on its associated roboticgripper's fingers, and is configured (e.g., by including a USB or othertransceiver circuit) to transmit the tactile information to one or bothof the arm control circuit and the interface computer. By configuringeach gripper controller to generate separate tactile information andfinger control signals, the present invention facilitates generating andrecording physical parameter values in an encoded format while alsoallowing the gripper controller to directly control the gripper fingers(i.e., by generating the finger control signals in a format thatdirectly causes the finger actuators to perform an accident/preventionoperation. In a specific embodiment the arm control circuit isconfigured to transmit system gripper control signals to the gripper'sfinger actuators (i.e., in addition to other primary robot controlsignals including the arm control signals transmitted to the armactuators) during a first “normal” operating period (e.g., while currentsensor data fails to match at least one stored set of indicatorparameter values), and the finger control signals generated by thegripper controller form at least a portion of the secondary robotcontrol signals generated when a potential imminent accident/failure isdetected (i.e., when the measured physical parameter values match anassociated set of stored indicator parameter values). By configuring thearm controller to generate primary robot control signals during normalrobot operating periods and configuring the gripper controller togenerate secondary robot control signals that effectively override(supersede) the primary robot control signals when a potential imminentaccident/failure is detected (i.e., by way of transmitting the fingercontrol signals directly to the finger actuators), the present inventionboth enhances normal operating efficiency while providing reducedlatency to prevent or mitigate the potential accident/failure.

In some embodiments each work cell includes at least one of arange/distance (e.g., radar) sensor configured to generate range dataand one or more cameras configured to generate image data (visual input,e.g., RGB and/or depth), where the range data and/or image data is usedto supplement the measured physical parameter values provided by themultimodal sensor arrays. In some embodiments, at least one external(first) camera is disposed external to the work cell's robotic systemand one or more gripper-mounted (second) cameras are disposed on therobotic system (e.g., the robot arm and/or the robotic gripper). In apresently preferred embodiment, up to ten external cameras areconfigured to generate wide angle (first) robot operation image datathat is passed to the gripper/arm controllers for purposes ofcoordinating gripper operations and/or provided to the interfacecomputer for transmission to the cloud server. The gripper-mountedcameras are configured to generate close-up (second) image data showinginteractions between the gripper's fingers and target objects that ispassed to the gripper controller for processing with the measuredphysical parameter values (sensor data) received from the multimodalsensor arrays and the range data received from the range sensors. Byproviding image data and/or range data in addition to the measuredphysical parameter values, each work cell is further enabled to reliablyrecord all information required to diagnose possible robotic systeminefficiencies (e.g., to accurately diagnose the causes of robotoperation accidents/failures).

In alternative specific embodiments each multimodal sensor array isdisposed on an associated gripper finger structure and includes a set ofmultiple sensors including an array of pressure sensors and one or moreadditional sensors, where each of the pressure sensors and additionalsensors is configured to generate an associated single-sensor data(measured physical parameter) value. In an exemplary embodiment, the setof sensors of each multimodal sensor array includes a two-dimensionalarray of pressure sensors and one or more of (i) one or more temperaturesensors, (ii) one or more vibration sensors and (iii) one or moreproximity sensors. In the presently preferred embodiment, the pressuresensors are piezoelectric-type sensors including piezoelectric ceramicmaterial (e.g., lead zirconate titanate (PZT)) structures sandwichedbetween solder flux layers and electrodes (contact pads) formed onopposing surfaces of two PCB stack-ups, and is further enhanced byimplementing a Faraday cage that shields each PZT structure fromelectronic noise. In other embodiments the pressure sensors may beimplemented using one or more other piezoelectric materials or sensortypes, such as strain gauges, capacitive pressure sensors, cavity-basedpressure sensors, piezoresistive sensors or piezoelectric sensors, andthe pressure sensors forming each pressure sensor array may be arrangedin a symmetric pattern or an asymmetric/random pattern. In alternativeembodiments the one or more additional sensors include one or moretemperature sensors comprising resistive temperature detectors (RTD),thermoelectric, or other variants, one or more vibration/texture sensor(e.g., either piezoelectric/piezoresistive or MEMS-based sensorconfigured to detect vibrations, and at least one proximity sensor(e.g., a capacitive-coupling-type sensing element). In alternativeembodiments, these additional sensor(s) are fabricated/mounted on thesame PCB stack-up as the pressure sensors and/or disposed on a differentPCB stack-up from the pressure sensors.

In a presently preferred embodiment the robotic system includes multiplemultimodal sensor arrays, where each multimodal sensor array is disposedon an associated finger structure of the system's gripper. To facilitatethe efficient transfer and processing of the resulting large amount ofsensor data, each multimodal sensor array includes an associated sensordata processing circuit that receives the single-sensor sensor data(measured physical parameter) values from the multimodal sensor array'sset of multiple sensors over multiple parallel signal lines, andtransmits the single-sensor sensor data values to the gripper controllerin the form of a finger-level sensor data signal. In a preferredembodiment the multiple sensors and the associated sensor dataprocessing circuit are integrally connected to (i.e., respectivelymounted onto) a shared printed circuit board (PCB) structure. Thissensor-array-and-processing arrangement allows the multimodal sensorarrays to efficiently transmit measured physical parameter values to thegripper controller and/or the arm controller using a serial data bus,which in turn simplifies the associated controller's task of processingdifferent sensor-type data received from multiple multimodal sensorarrays disposed on various gripper fingers in order to quickly identifypotentially harmful or dangerous operating conditions. In a specificembodiment the sensor data processing circuit of each multimodal sensorarray is configured such that the single-sensor data values from each ofthe individual sensors are transmitted in parallel through associatedsignal conditioning (pre-processed) circuits to a sensor data collectioncircuit, which then transmits the collected values on a serial data lineto a finger-level generation circuit for conversion into a correspondingfinger-level sensor data signal, which is then transmitted on a serialsignal line to the gripper controller. By providing multiple multimodalsensor arrays on each gripper finger, the work cells of the presentinvention provide rich sensory tactile data that facilitates theefficient development of optimal robot operating processes. In addition,by combining the multiple sensors of each multimodal sensor array withan associated sensor data processing circuit, the present inventionfacilitates the transfer of large amounts of sensor data over arelatively small number of signal lines.

In some embodiments the operating systems utilized by the controllerand/or interface computer to perform robot operations utilizes one ormore AI models. Such AI models are utilized to both augment each robotsystems' capabilities beyond simple pre-programmed repetitive tasks, andalso to react/adapt to new situations. In such embodiments the operatingsystem updates transmitted to each work cell from the cloud computer mayinclude updated AI models that have been developed by analyzing thetactile information and other data collected from all of the work cells.

In one embodiment the network is configured to facilitate an arrangementin which the cloud server is controlled and operated by a robot systemservice provider, and the work cells are utilized by different cliententities having a contractual arrangement with the robot system serviceprovider. To facilitate this arrangement such that tactile informationand other sensor data generated by each work cell is readily availablefor transmission to the cloud server while the security (privacy) ofproprietary information owned by the client entities is maintained, boththe cloud server and each work cell are configured to utilize at leasttwo separate databases: a first database that includes shareddata/information, and a second database that includes restricted orproprietary data/information. Specifically, each work cell is configuredto include two databases that are either integrated into or otherwiseaccessible by way of the work cell's interface computer: a network(first) database and a local (second) database. Similarly, the cloudserver is configured to utilize restricted (first) work cell informationdatabase and a shared data (second) sensor data/master operating systemupdate database.

In an exemplary embodiment the network database implemented by each workcell is configured to store data that is written to or read from thecloud server via the network bus and the work cell's interface computer.That is, the network database is configured to allow the cloud server toboth perform automatic operating system updates and automatic downloadsof sensor and work cell operating data from each work cell on areal-time or delayed synchronization basis. For example, operatingsystem updates (including AI model updates) and optional updatedtraining features transmitted from the cloud server are stored in thenetwork database of each work cell. In addition to the sensor data(mentioned above), additional shared data/information that may be storedby each work cell for transfer to the network database may includegrasping performance information, number of grasps performed, number ofsuccessful grasps, ongoing maintenance related data, bug reports andother general operating data that may be used to facilitate theefficient diagnosis of problems and the identification of otherservice-related issues associated with each work cell. In oneembodiment, such additional shared data/information is transformed in aprivacy preserving manner before being stored on the network database.

In another exemplary embodiment the local (second) database of each workcell is configured to store proprietary data owned by the work cellowner/operator (i.e., the network client). Such proprietary data mayinclude a specific sequence of actuator signals developed by a givenwork cell's owner/operator that provide the owner/operator a competitiveadvantage over other work cell owners/operators. In one embodiment theproprietary data stored in each work cell database is protected fromaccess by the cloud server. In another embodiment, each work celldatabase may be configured to allow the cloud server restricted access(i.e., limited in a privacy preserving manner) to some or all of theproprietary data.

In other embodiments work cell (first) database utilized by the cloudserver is configured to store specific work cell-related information forall of the network's work cells, and the sensor data/OS-update masterfile (second) database that is configured to store measured physicalparameter values and other data collected from the work cells foranalysis, along with and a master file of updated improved/updatedoperating system software that is transmitted to the work cells on areal-time delayed synchronization basis. In an exemplary embodiment, thework cell information database is configured to store data associatedwith the robot mechanism configuration and operating history for eachwork cell (e.g., specific or custom configuration features, number ofoperations performed by each structure and actuator of the associatedrobot mechanism, etc.), and this information is utilized by the cloudserver to determine, for example, which accident/failure avoidanceoperations from the master list are applicable to a given work cell. Forexample, some accident/failure avoidance operations may apply to onespecific configuration (e.g., systems configured to usethree-fingered-gripper mechanisms) and not to alternative configurations(e.g., systems configured to use two-fingered-gripper mechanisms), andother accident/failure avoidance operations may apply only to robotmechanisms having grippers that have performed 10,000 graspingoperations without service. In another exemplary embodiment, the sensordata/OS-update master file database may include, for example, a masterlist of accident/failure avoidance operations (e.g., one or more sets ofmeasured physical parameter values that indicate the potential imminentreoccurrence of one or more associated robotic accidents/failures, alongwith associated secondary robot control signals that, when performed bya robot mechanism, prevent the associated robotic accident/failures'reoccurrence). During an exemplary database synchronization processinvolving a specific work cell, the cloud server identifies a group ofaccident/failure avoidance operations that are applicable to a givenwork cell's specific configuration, the cloud server verifies that thegiven work cell's system database includes a copy of the identifiedgroup, and automatically synchronizes (i.e., transmits over the networkbus) the work cell's system database to include any new or updatedaccident/failure avoidance operations on a real-time or delayedsynchronization basis.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects and advantages of the presentinvention will become better understood with regard to the followingdescription, appended claims, and accompanying drawings, where:

FIG. 1 is a diagram depicting an exemplary robotic network including acloud server and multiple robot systems according to a generalizedembodiment of the present invention;

FIG. 2 is a simplified circuit diagram showing a robotic networkaccording to a specific embodiment of the present invention;

FIG. 3 is a partial perspective view of an exemplary gripper utilized ina work cell of the network of FIG. 2 according to a specific embodimentof the present invention;

FIG. 4 is a simplified circuit diagram showing a multimodal sensor arrayformed on a robot gripper finger according to another specificembodiment of the present invention; and

FIG. 5 is a simplified circuit diagram showing a gripper including themultimodal sensor array of FIG. 4 and additional circuitry according toanother specific embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS

The present invention relates to a computer-based method for managing anetwork of similar robotic systems in a way that greatly enhances thecapabilities of the networked robotic systems. The following descriptionis presented to enable one of ordinary skill in the art to make and usethe invention as provided in the context of a particular application andits requirements. As used herein, directional terms such as “upper”,“lower”, “lowered”, “front” and “back”, are intended to provide relativepositions for purposes of description and are not intended to designatean absolute frame of reference. With reference to electrical connectionsbetween circuit elements, the terms “coupled” and “connected”, which areutilized herein, are defined as follows. The term “connected” is used todescribe a direct connection between two circuit elements, for example,by way of a metal line formed in accordance with normal integratedcircuit fabrication techniques. In contrast, the term “coupled” is usedto describe either a direct connection or an indirect connection betweentwo circuit elements. For example, two coupled elements may be directlyconnected by way of a metal line, or indirectly connected by way of anintervening circuit element (e.g., a capacitor, resistor, inductor, orby way of the source/drain terminals of a transistor). Variousmodifications to the preferred embodiment will be apparent to those withskill in the art, and the general principles defined herein may beapplied to other embodiments. Therefore, the present invention is notintended to be limited to the particular embodiments shown and describedbut is to be accorded the widest scope consistent with the principlesand novel features herein disclosed.

FIG. 1 shows an exemplary robotic network 80 configured to facilitatenetwork communication events involving data transmissions between acloud server 90 and multiple work cells 100-1, 100-2 . . . 100-N, whichare collectively referred to below as “work cells 100-1/2 . . . N”.

Exemplary network communication events are described below in a greatlysimplified form as data transmissions (electronic messages) sent over anetwork bus 85 to which the cloud server 90 and work cells 100-1/2 . . .N are respectively operably coupled. In a practical embodiment, robotnetwork 80 is configured as a software-defined networking in a wide areanetwork (SD-WAN) in which cloud server 90 and work cells 100-1/2 . . . Nrespectively implement secure communication hardware (e.g., Fortigate60F gateway/router/firewall appliances), and network communicationevents (data transmissions) are encoded for privacy using a selectedvirtual private network (VPN) protocol (e.g., IPsec VPN) and transmittedover network bus 85, which in this case is the Internet. In otherembodiments, network bus 85 may be implemented using a local areanetwork and/or wireless data transmissions. Note that the networkcommunication events mentioned below are greatly simplified for brevityin view of these alternative networking configurations. For example,practical secure transmission hardware and other details associated withestablishing such SD-WAN-type network arrangements are known to thoseskilled in the networking arts, and are therefore omitted from thefigures and following description, but this omission is not intended tobe limiting unless specified in the appended claims.

In the following description all network communication events involvepoint-to-point transmissions between cloud server 90 and one of workcells 100-1/2 . . . N (that is, work-cell-to-work-cell transmission arenot supported by robotic network 80). For example, a first networkcommunication event may involve an updated OS transmitted from cloudserver 90 to work cell 100-1, a second event may involve a separatetransmission of the updated OS from cloud server 90 to work cell 100-2,and a third event may involve a data transmission from work cell 100-1to cloud server 90. Although the network communication events describedherein are point-to-point, this data transmission arrangement is notintended to be limiting. For example, network bus 85 may be implementedusing a local area network and/or wireless data transmissions, androbotic network 80 may support work-cell-to-work-cell transmissions, forexample, by way of forming one or more sub-networks respectivelyincluding two or more work cells).

Referring to the upper left portion of FIG. 1, cloud server 90 functionsas a network host by way of collecting tactile information TI (whichincludes measured physical parameter values, as described below) from atleast some of work cells 100-1/2 . . . N, and by automaticallytransmitting operating system updates OSU to work cells 100-1/2 . . . Nfor reasons explained below. Note that tactile information TI isgenerated in accordance with robotic operations performed by each workcell's robotic system, and each work cell is typically utilized indifferent ways and at different times from the other work cells.Accordingly, even though each work cell is similarly configured andgenerates tactile information based on similarly measured physicalparameter values (e.g., the temperature of a grasped target object), themeasured physical parameter values tactile information TI transmittedfrom a given work cell (e.g., work cell 100-1) is almost alwaysdifferent from the measured physical parameter values included intactile information TI received from another work cell (e.g., work cell100-2). Accordingly, correlating the tactile information TI receivedfrom multiple work cells 100-1/2 . . . N provides a more efficient andreliable basis for addressing non-optimal processes (e.g., identifyingboth conditions that give rise to an accident, and ways to improve theoperating system to prevent reoccurrences of the accident) than thatgenerated by tactile information from a single work cell.

In one embodiment robotic network 80 is configured to facilitate anarrangement in which the cloud server 90 is controlled and operated by ahost entity that is a robot system service provider, and work cells100-1/2 . . . N are utilized by different client entities having acontractual arrangement with the robot system service provider. Tofacilitate this arrangement, cloud server 90 is configured to utilizerestricted (first) work cell information (WORK CELL INFO) database 95and a shared data (second) sensor data/master operating system update(SENSOR DATA/MASTER OSU) database 97. Work cell information database 95is configured to store specific work cell-related information (e.g.,configuration and model information, service information such as generalsystem health, number of actions performed, etc.) for each work cell100-1/2 . . . N, and the sensor data/OS-update master file database 97is configured to store measured physical parameter values (e.g., tactileinformation TI) and other data (e.g., visual input) collected from thework cells for analysis, along with and a master file of operatingsystem updates OSU that are transmitted from cloud server 90 to workcells 100-1/2 . . . N on a real-time delayed synchronization basis. Inan exemplary embodiment, work cell information database 95 is configuredto store work cell information WCI, which includes data associated withthe robot mechanism configuration and operating history for each workcell 100-1/2 . . . N (e.g., specific or custom configuration features,number of operations performed by each structure and actuator of theassociated robot mechanism, etc.), and sensor data/OS-update masterdatabase 97 may include, for example, a master list of accident/failureavoidance operations (e.g., one or more sets of measured physicalparameter values that indicate the potential imminent reoccurrence ofone or more associated robotic accidents/failures, along with associatedsecondary robot control signals that, when performed by a robotmechanism, prevent the associated robotic accident/failures'reoccurrence).

Work cells 100-1/2 . . . N of robotic network 80 respectively include aninterface computer and one or more robotic systems, where each roboticsystem includes one or more robot mechanisms having an associatedcontrol circuit. For example, referring to the upper right portion ofFIG. 1, work cell 100-1 includes an interface computer 150-1 and arobotic system 200-1, where robotic system 200-1 includes a robotmechanism 201-1 and a control circuit 203-1. Similarly, referring to thelower portion of FIG. 1, work cell 100-2 includes an interface computer150-2 and a robotic system 200-2 comprising a robot mechanism 201-2 anda control circuit 203-2, and work cell 100-N includes an interfacecomputer 150-N and a robotic system 200-N comprising a robot mechanism201-N and a control circuit 203-N. In a presently preferred embodiment,robot mechanisms 201-1 to 200-N of robotic systems 200-1 to 200-N havesimilar or identical configurations. Therefore, for brevity, theconfiguration and operation of work cells 100-1/2 . . . N is describedbelow with reference to interface computer 150-1 and robotic system200-1 of work cell 100-1 (i.e., the following description of interfacecomputer 150-1, robot mechanism 201-1 and control circuit 203-1effectively describes operations performed by corresponding elements ofwork cells 100-2 to 100-N).

Referring again to work cell 100-1, interface computer 150-1 isconfigured to facilitate two-way communications between control circuit203-1 of robotic system 200-1 and cloud server 90 (e.g., by way of datatransmissions over network bus 85), and also configured to performadditional functions related to the operation of work cell 100-1. In thedepicted exemplary embodiment, interface computer 150-1 includestransceiver and other hardware circuitry (not shown) that is operablyconfigured to receive an operating system update OSU from cloud server90 over network bus 85, and configured to transmit tactile informationTI to cloud server 90 over network bus 85 (note that operating systemupdate OSU and tactile information TI are described below). Interfacecomputer 150-1 is also configured to transfer received operating systemupdate OSU to control circuit 203-1 for reasons described herein and isalso configured to receive tactile information TI from robotic system200-1. In a presently preferred embodiment interface computer 150-1 isalso configured as a local input/output device that facilitates theentry of user-designated instructions UDI by a user/owner, whereuser-designated instructions UDI represent a generalized form ofinstructions utilized to control robotic system 200-1 such that itperforms the user/owner's desired specific robot operation.

As indicated in FIG. 1, in one embodiment work cells 100-1/2 . . . Nrespectively include a network database (NET DB) 160-1/2 . . . N and alocal database (LOCAL DB) 170-1/2 . . . N that are implemented by memorycircuits integrated into each work cell's interface computer 150-1/2 . .. N, or are implemented by external storage devices that are accessibleby an associated interface computer 150-1/2 . . . N. That is, interfacecomputers 150-1/2 . . . N are configured to format and utilizeassociated network databases 160-1/2 . . . N and associated localdatabases 170-1/2 . . . N in the manner described below.

Network databases 160-1/2 . . . N function to facilitate the uploadtransfer of measured physical parameter values from work cells 100-1/2 .. . N to cloud server 90, and to optionally also facilitate theautomatic download transmission of new operating system updates OSU tocontrol circuits 203-1/2 . . . N. Network databases 160-1/2 . . . Nfacilitate the transfer of measured physical parameter values by way ofstoring accrued tactile data TI (including measured physical parametervalues) in a way that allows cloud server 90 to both upload and download(read/transfer and store) the accrued tactile information TI into sensordata/master OSU database 97. The storage of tactile data TI in networkdatabases 160-1/2 . . . N is performed by associated interface computers150-1/2 . . . N, which are respectively configured to receive tactiledata TI either directly from an associated multimodal sensor array260-1/2 . . . N or by from an associated control circuit 203-1/2 . . .N. Network databases 160-1/2 . . . N also optionally facilitateautomatic operating system updates by way of storing operating systemupdates OSU transmitted from cloud server 90 and other public data(e.g., updated training features). For example, network database 160-1is configured to store tactile information TI (i.e., measured physicalparameter values PPV and other data generated during the operation ofrobotic system 200-1), thereby facilitating automatic uploading by cloudserver 90 on a real-time or delayed synchronization basis for storage insensor data/master OSU database 97. In a similar manner, operatingsystem updates OSU that are automatically transmitted to work cell 100-1from cloud server 90 may be stored in network database 160-1concurrently with or before being implemented by control circuit 203-1.In addition to tactile information TI, network databases 160-1/2 . . . Nmay be utilized to store additional shared work-cell-specificdata/information that is generated during the operation of roboticsystems 200-1/2 . . . N and subsequently uploaded to cloud server 90.For example, control circuit 203-1 may be programmed to monitor variousevents that occur during robot operations performed by robotic system200-1 (e.g., number of grasps performed by gripper 260-1), and to recordthis data/information, which is specific to work cell 100-1, in networkdatabase 160-1 for subsequent transfer by cloud server 90 to work celldatabase 95. Such work-cell-specific information may include, forexample, grasping performance information, number of grasps attempted,number of successful grasps, ongoing maintenance related data, bugreports and other general operating data that may be used to facilitatethe efficient diagnosis of problems and the identification of otherservice-related issues associated with the maintenance of each work cell100-1/2 . . . N. In one embodiment, such additional shareddata/information is transformed in a privacy preserving manner byinterface computers 150-1/2 . . . N before being stored on an associatednetwork database 160-1/2 . . . N.

In an exemplary embodiment local databases 170-1/2 . . . N of work cells100-1/2 . . . N are respectively configured to store proprietary data ofeach work cell's owner/operator (e.g., the network client) in a way thatrestricts or prevents access by cloud server 90. For example, in someembodiments proprietary data owned by the owner/operator of work cell100-1 (e.g., a specific sequence of robot arm movements is stored inwork cell database 170-1, which is configured to prevent free access bycloud server 90. In other embodiments, each work cell database 170-1/2 .. . N may be configured to allow the cloud server 90 restricted access(i.e., limited in a privacy preserving manner) to some or all of theproprietary data that might be utilized by a network service provider tofurther broaden the learning base for purposes of developing improvedrobotic motion and grasping models (e.g., by utilizing the restrictedproprietary data to run both virtual and real-world training exercisesdesigned to generate operational statistics that compare data from thevarious deployed work cell configurations to gain insights as to whichconfigurations perform best in which circumstances). Of course, anyproprietary data that might be accessed by the cloud server is protected(i.e., not synchronized with other data that might be provided to thenetwork's client work cells) and is only applied towards new modelsdeveloped for the client work cells.

In a preferred embodiment the robotic systems 200-1/2 . . . N of workcells 100-1/2 . . . N have the same or a similar configuration so thattactile information TI generated by one robotic system may bebeneficially used to improve the robotic operations performed by allrobotic systems 200-1/2 . . . N. As indicated in work cell 100-1,robotic system 200-1 generally include a robotic arm (robot mechanism)201-1 having a fixed end 211-1 and a gripper (end effector) 260-1connected to the opposing distal (free) end. Robot operations performedby robotic system 200-1 involve controlling the mechanical movements ofrobotic arm 201-1 and gripper 250-1 using control signals transmittedfrom associated control circuit 203-1 such that, for example, gripper250-1 is caused to grasp and move a target object OBJ. To facilitate thecollection of enhanced multimodal sensor data (i.e., tactile informationTI), robotic system 200-1 includes one or more multimodal sensor arrays260-1 (shown in bubble section in upper left portion of FIG. 1), whereeach sensor array includes different types of sensors 261 thatrespectively measure associated physical parameter values (e.g., appliedgripper pressure, target object temperature and other forms of tactileinformation). Control circuit 203-1 includes (i.e., is configured tooperate in accordance with) an operating system OS (i.e., a softwareprogram executed by one or more processors, which may include an AI ormachine learning model) that it uses during normal operations to convertuser-designated instructions UDI into corresponding primary robotcontrol signals PRCS, which in turn cause the various actuators A ofrobot mechanism 201-1 to perform the owner/user's desired specific robotoperations.

As mentioned above, in a presently preferred embodiment robot mechanism201-1 is an arm-type mechanism (robot arm) made up of series-connectedrobot arm structures that are fixedly attached at one end to a basestructure and are operably connected to gripper 250-1 (or other endeffector) at an opposing distal (free) end. In the simplified embodimentdepicted in FIG. 1, fixed base mechanism 211-1 of robot mechanism 201-1is attached to a table-type base structure 110-1, and includes an upperarm structure 215-1 extending from base mechanism 211-1 to an elbowmechanism 220-1, a forearm structure 225-1 extending from elbowmechanism 220-1 to a wrist mechanism 230-1, and a robot gripper (aka,gripper mechanism, gripper or gripper-type end effecto) 250-1 attachedto a distal end of wrist mechanism 230-1. Robot mechanism 201-1 alsoincludes multiple actuators A (only three are shown for brevity), eachactuator including a motor control circuit (not shown) configured toturn on/off one or more associated electric motors (not shown) inresponse to control signals received from control circuit 203-1, therebycausing the interconnected structures forming arm-type robot mechanismto move relative to each other. In the depicted exemplary embodiment, afirst actuator 216-1 is disposed in base mechanism 211-1 to facilitateselective rotation and pivoting of upper arm structure 215 relative tobase table 110-1, a second actuator 216-1 is disposed in elbow mechanism220-1 and configured to facilitate selective pivoting of forearmstructure 225-1 relative to upper arm structure 215-1, a third actuator226-1 is disposed in wrist mechanism 230-1 to facilitate selectivepivoting of gripper 250-1 relative to forearm structure 225-1, and afourth actuator 256-1 is disposed in gripper 250-1 that controlopening/closing of the gripper's fingers to facilitate grasping andreleasing a target object OBJ. Exemplary robot mechanism 201-1 is merelyintroduced to provide a simplified context for explaining the featuresand benefits of the present invention, and the configuration ofexemplary robot mechanism 201-1 is not intended to limit the appendedclaims.

Referring to the bubble section provided in the upper-left portion ofFIG. 1, gripper mechanism (gripper or gripper-type end effector) 260-1includes at least one multimodal sensor array 260-1 having multiplesensors (or sets of sensors) 261 that are respectively configured tomeasure corresponding physical parameter values during each roboticoperation involving an interaction between gripper 250-1 and targetobject OBJ. Gripper 250-1 includes various mechanisms and structuresthat are operably configured in accordance with known techniques tofacilitate gripper operations involving moving two or more opposinggripper fingers to grasp and manipulate target object OBJ during robotoperations. As indicated in the bubble section, multimodal sensor array260-1 is disposed on a corresponding gripper finger 254-1 and includessensors 261 that are respectively operably configured to generate anassociated set of measured physical parameter values PPV in response tointeractions between gripper finger 254-1 and target object OBJ, andmultimodal sensor array 260-1 is configured to transmit measuredphysical parameter values PPV (e.g., in the form of tactile informationTI, which is generated as described below) to control circuit 203-1 orto interface computer 150-1 during each robotic operation.

Control circuit 203-1 is configured to control robot mechanism 201-1during robot operations by generating control signals in accordance withuser-designated instructions UDI and/or an operating system OS duringrobot operations, and by transmitting the control signals to theactuators of robot mechanism 201-1 in a sequence defined byuser-designated instructions UDI and/or operating system OS. In oneembodiment, control circuit 203-1 includes processor circuitry that isprogrammed (controlled) by way of (i.e., is configured to operate inaccordance with) an operating system OS (i.e., a software program) thatis configured during normal operations to convert the user-designatedinstructions UDI into corresponding primary robot control signals PRCSthat cause the various actuators of the robot mechanism 201-1 to performdesired specific robot operations. In some embodiments, operating systemOS may include an artificial intelligence (AI) or machine learning modelconfigured to augment robotic system capabilities beyond simplepre-programmed repetitive tasks defined by user-designated instructionsUDI (e.g., to react to new situations).

According to an aspect of the present invention, cloud server 90 isconfigured to receive tactile information TI, which includes at leastsome of the measured physical parameter values generated by themultimodal sensor arrays of work cells 100-1/2 . . . N (e.g., measuredphysical parameter values PPV generated by array 260-1 of work cell100-1), and is configured to automatically transmit operating systemupdates OSU to control circuits 203-1/2 . . . N of work cells 100-1/2 .. . N. In one embodiment, tactile information TI received from two ormore of work cells 100-1/2 . . . N is utilized to diagnose non-optimalrobotic operations by the network host/service provider, and newoperating system updates OSU that are generated to improve robotoperations are automatically transmitted (downloaded) by cloud server 90to control circuits 203-1/2 . . . N of work cells 100-1/2 . . . N (i.e.,by way of network bus 85 and interface computers 150-1/2 . . . N). Thatis, before a given automatic update (i.e., before a given operatingsystem update OSU is downloaded to any of work cells 100-1/2 . . . N),each control circuit 203-1/2 . . . N generates control signals inaccordance with its currently implemented (initial/non-updated) versionof operating system OS, and after the given automatic update (i.e.,after the given operating system update OSU is downloaded to all of workcells 100-1/2 . . . N), each control circuit 203-1/2 . . . N generatescontrol signals in accordance with the newly updated operating system(i.e., any improvements to the OS provided with the given operatingsystem update OSU are only implemented after the automatic update).

FIGS. 2 and 3 depict a simplified robotic network 80A according toanother embodiment including a work cell 100A connected to cloud server90A by way of network bus 85. Cloud server 90A and network bus 85 areconfigured and operate essentially as described above with reference toFIG. 1. Work cell 100A is intended to represent one of multiple similarwork cells coupled to cloud server 90A in the manner described abovewith reference to FIG. 1. Work cell 100A is similar to work cell 100-1(FIG. 1) in that it includes an interface computer 150A and a roboticsystem 200A, where robotic system 200A includes an arm-type robotmechanism (RM) 201A and a control circuit 203A. Although not specifiedin FIG. 2, robot mechanism 201A is understood to include grippermechanism 250A, which is shown in FIG. 3. Various features of work cell100A that differ from corresponding portions of work cell 100-1 (FIG. 1)are described in detail below. All other portions of work cell 100A thatare not specifically mentioned below are understood to operate in amanner consistent with corresponding portions described above withreference to FIG. 1. For example, cloud server 90A functions toautomatically upload tactile information TI from system database 160Aand to automatically download operating system updates OSU to controlcircuit 203A, and interface computer 150A functions to facilitate theupload/download transmissions and to facilitate the generation ofuser-defined instructions UDI that are passed to control circuit 203A asdescribed above.

According to a first feature difference of work cell 100A, controlcircuit (CC) 203A is implemented using two separate controller (e.g.,microprocessor) circuits: a robot arm controller 203A-1 and a grippercontroller 203A-2. Robot arm controller 203A-1 is mounted in the mannerdescribed above with reference to control circuit 203-1 (e.g., on atable or other base structure), and as indicated in FIG. 3, grippercontroller 203A-2 is mounted on gripper 260A. In the exemplaryembodiment shown in FIG. 2, robot arm controller 203A-1 is configured byway of a current operating system (or a portion thereof) to generateprimary robot control signals PRCS during normal robotic operations,where primary robot control signals PRCS include both arm controlsignals ACS, which are transmitted to arm actuators 212A, and systemgripper control signals SGC, which are transmitted to finger actuators256A. In contrast, gripper controller 203A-2 is configured by way of thecurrent operating system (or a portion thereof) to receive and processtactile information TI in the manner described below during normaloperations, and to generate one or more secondary robot control signalsSRCS when the tactile information indicates an abnormal operatingcondition. That is, during normal operations all control signalsutilized to control the movements of robot arm mechanism 201A andgripper 260A are generated by robot arm controller 203A-1, and grippercontroller 203A-1 only functions to receive and process tactileinformation TI in the manner described below. Conversely, when tactileinformation TI processed by gripper controller 203A-1 indicates anabnormal operating condition, gripper controller 203A-2 generates one ormore finger control signals FC that cause gripper 250A to perform apredefined gripping operation (e.g., to close/grasp, open/release orapply a modified gripping pressure to a target object).

The operations performed by robot arm controller 203A-1 and grippercontroller 203A-2 will now be described with reference to a greatlysimplified rules-based-programming example involving thecold-object-drop accident, which is introduced in the background section(above). As set forth above, the proposed methodology implemented by arobotic network of the present invention involves utilizing multimodalsensor arrays (e.g., array 260B of robotic system 200A) from multiplework cells to collect and correlate measured physical parameter valuesPPV, which in this example include the temperature of grasped objectsand the applied grasping pressure. One skilled in the art wouldunderstand that correlating this temperature and pressure data couldfacilitate identifying the underlying cause of the accidents (e.g.,objects are dropped when their temperature is below 10° C.). Inaddition, once the accident's cause is identified, a suitable correctiveaction may be devised that, when performed by each work cell, preventsreoccurrence of the cold-object-drop accident. Based on this diagnosis,a corresponding rules-based operating system update OSU is thengenerated that includes, for example, indicator parameter valuessuitable for detecting the potential imminent reoccurrence of thecold-object-drop accident (e.g., the indicator parameter values mayinclude a check for currently measured object temperature values equalto or below 10° C.), and also includes one or more correspondingsecondary robot control signals SRCS that prevent or mitigate theaccident (e.g., one or more finger control signals FC that cause fingeractuators 256A to increase gripper pressure to twelve kPa when currentlymeasured physical parameter values match the stored indicator parametervalue). After being transmitted to work cell 100A by the cloud server90A, this operating system update OSU modifies the performance ofcontrol circuit 203A during subsequent robot operations, for example, bycausing gripper controller 203A-2 to store the indicator parameter value(i.e., object temperature greater than or equal to ten degrees Celsius),to compare the stored value with currently measured temperature valuesgenerated by temperature sensor(s) of multimodal sensor array 260A, andto increase the gripper pressure to twelve kPa (e.g., by transmittingfinger control signals FC to gripper actuator 256A) whenever a currentlymeasured temperature value is at or below 10° C., thereby preventingreoccurrence of cold-object-drop accidents.

The example set forth above is intended to illustrate how an exemplaryoperating system update OSU may be generated using a “rules based”programming approach, where stored set of indicator parameter values IPVand associated secondary robot control signal SRCS represent adifference (improvement) between an initial (non-updated) version of theoperating system implemented by control circuit 203A prior to receivingthe OSU transmission from cloud server 90, and the updated operatingsystem implemented by control circuit 203A after the transmission. Thatis, before the update, control circuit 203-1 generates control signalsin accordance with the earlier/initial (non-updated) version ofoperating system OS (i.e., control circuit 203-1 generates primarycontrol signal PCRS using gripper control signal SGC and arm controlsignal ACS even when multimodal sensor arrays 260A detect target objectshaving temperatures of 10° C. or lower, whereby the intermittentcold-object-drop accident may be repeated). Conversely, after receivingand implementing operating system update OSU, control circuit 203-1generates secondary control signal SCRS whenever multimodal sensorarrays 260A detect target objects having a temperatures of 10° C.,whereby the increased gripper pressure reliably prevents thereoccurrence of the cold-object-drop accident. In other embodiments thatimplement non-rules-based programming methods, the measured physicalparameter values generated by multimodal sensor arrays 260A are alsoused as indicators, but control of the robot may be decided by anynumber of additional methods including heuristics that use motionprimitives, or “black box” approaches that use neural networks toprovide mapping between parameter inputs and action outputs. Statisticalmethods or machine learning may also be used to predict environmental orsystem states that might be relevant to direct or even indirect controlof hardware and operating systems. An example of direct control is onewhere sensor data are used as inputs to an algorithm or machine learningmodel producing commands to a robot controller as output, while indirectcontrol might use a machine learning model to predict where an object ofinterest is in real-world coordinate space, and use that value as inputto a secondary model that sends commands to the robot mechanism. So,while the sensor data generated by the multimodal sensor arrays of thepresent invention can be useful for dynamic (e.g., on-gripper) responsesto events such as anomalies, they can also be used to build predictivemodels where data are used to update model parameters rather thandirectly inform an interpretable action. In this context, algorithms orstatistical methods comprising machine learning applications generallyrequire data collection from physical systems which are stored,processed and used to learn a model for prediction. Downstream taskssuch as storage, processing and training for machine learning may occurdirectly on hardware that performs collection locally, or in remoteclusters, such as the cloud. Here, storage refers to any way ofcapturing data to be accessed later in time, while processing usuallydescribes methods of data engineering that are required before modeltraining can occur, and as part of data management or transport. Thisincludes operations such as transforming data between types, applyingmathematical functions, and generating derivative data representationsthrough annotation or auxiliary machine learning pipelines. Training isthe process of learning optimized model parameters by iterativelyupdating parameters based on raw or processed data. By combining notonly raw data from multi-modal sensors, but derived data representationscreated by the described methods across a network of robotic work cells,the global properties of a behavior model can be optimized moreefficiently. New capabilities can be deployed to a single work cell or anetwork of work cells, and data captured can go back into the “blackbox” pipeline for further optimization and improvement. Referring againto FIG. 2, work cell 100A is further distinguished in that it includesat least one external camera 180A disposed external to the work cell'srobotic system 200A, and at least one range/distance (e.g., radar)sensor 270A and one or more gripper-mounted (second) cameras 280A. Asindicated in FIG. 3, range/distance sensor 270A and gripper-mountedcameras 280A are disposed on robotic gripper 250A. In a presentlypreferred embodiment, up to ten external cameras 180A are configured togenerate wide angle (first) image data ID1 that is passed to the controlcircuit 203A for purposes of coordinating gripper operations and/orprovided to interface computer 150A for transmission to cloud server90A. Range/distance sensor 270A is configured to generate range data RDindicating a measure distance between gripper 250A and a target object.Gripper-mounted cameras 280A are configured to generate close-up(second) image data ID2 showing interactions between the gripper'sfingers and target objects that is passed to the gripper controller203A-2 for processing with the measured physical parameter values(sensor data PPV) received from the multimodal sensor arrays 260A andthe range data RD received from range sensor 270A. In one embodimentrange data RD is transmitted directly to gripper controller 203A-2, butin other embodiments may be provided to arm controller 203A-1 and/orinterface computer 150A. In one embodiment, tactile information TIincludes measured physical parameter values PPV, range data RD and atleast one of image data ID1 and image data ID2, whereby range data RDand image data ID1 and/or 1D2 are used to supplement the measuredphysical values PPV provided by the multimodal sensor arrays 260B.

FIG. 3 shows an exemplary gripper 250A utilized by work cell 100A (FIG.2). Gripper 250A generally includes a gripper mechanism 251A, grippercontroller 203A-1 and finger actuators 256A. Gripper mechanism 251Aincludes a base/frame structure 252A, a connection fixture 253A that isfixedly connected to base structure 252A and configured to operablysecure the robotic gripper 250A to a distal end of robot arm mechanism201A, and gripper fingers 254A-1 and 254A-2 that are movably connectedto the base structure 252A. Finger actuator 256A is at least partiallydisposed on base structure 252A and including a motor/controller/encoderconfiguration that controls the open/close relative movement of gripperfingers 254A-1 and 254A-2 by way of finger (mechanical) linkages MC.Those skilled in the art understand that the various structures andmechanisms of robotic gripper 250A may be implemented in many ways usingtechniques known in the art, and that the description of grippermechanism 251A provided herein is greatly simplified to emphasize thenovel aspects of the present invention.

Referring to the left side of FIG. 3, multimodal sensor arrays 260A-1and 260A-2 are disposed (i.e., integrated into or otherwise mounted on)gripper mechanism 251A. Specifically, multimodal sensor array 260A-1 ismounted onto gripper finger 254A-1, and multimodal sensor array 260A-2is mounted onto gripper finger 254A-2 such that arrays 260A-1 and 260A-2are facing each other. Sensor array 260A-1 generally includes a sensorgroup (set of sensors) 261A-1 and a sensor data processing circuits263A-1, and sensor array 260A-2 includes a sensor group (set of sensors)261A-2 and a sensor data processing circuits 263A-2. Sensor groups261A-1 and 261A-2 are respectively fixedly attached to opposing contactsurface portions of gripper fingers 254A-1 and 254A-2 and positionedsuch that sensor groups 261A-1 and 261A-2 are disposed between a targetobject (not shown) and sensor data processing circuits 263A-1 and263A-2, respectively, during operable interactions between roboticgripper 250A and the target object. Each sensor group 261A-1 and 261A-2includes multiple sensors that respectively generate sensor data valuesin response to interactions between gripper fingers 254A-1 and 254A-2and corresponding surface characteristics of a target object during eachcorresponding robotic system operation. For example, when roboticgripper 250A is controlled to grasp a target object, a sensor of sensorgroup 261A-1 collects associated single-sensor data value SSD-1 thatreflects the interaction between a corresponding contact portion 254-11of gripper finger 254A-1 and an opposing surface region of the targetobject, and passes the collected single-sensor data values SSD-1 tosensor data processing circuit 263A-1. Similarly, each sensor of sensorgroup/set 261A-2 collects sensor data values that reflect interactionsbetween gripper finger 254A-2 and the target object, and passes thecollected single-sensor data value to sensor data processing circuit263A-2. In one embodiment, each sensor of sensor groups 263A-1 and263A-2 generates one associated single-sensor data value during eachsensor data generation cycle (i.e., during each predetermined period oftime, such as each one second time period). Accordingly, because eachsensor group/set 261A-1 and 261A-2 includes multiple sensors, multiplesingle-sensor data values are transmitted to sensor data processingcircuits 263A-1 and 263A-2 during each sequential cycle. Sensor dataprocessing circuits 263A-1 and 263A-2 are configured to generatecorresponding finger-level sensor data signals FSD-1 and FSD-2 inresponse to the multiple single-sensor data values respectively receivedfrom sensor groups 261A-1 and 261A-2. For example, sensor dataprocessing circuit 263A-1 is configured to generate finger-level sensordata signal FSD-1 in according with single-sensor data values SSD-1received from sensor group 261A-1. In one embodiment sensor dataprocessing circuit 263A-1 is configured to receive single-sensor datavalues SSD-1 in parallel from sensor group 261A-1 (i.e., such that eachsingle-sensor data values SSD-1 is received on a unique conductivesignal path extending from a corresponding sensor of sensor group 261A-1and a corresponding input terminal of sensor data processing circuit263A-1), and finger-level sensor data signal FSD-1 is generated as aserial data signal (i.e., such that all of finger-level sensor datasignal FSD-1 is transmitted over a single elongated conductor(conductive path) 255A-1 to gripper controller (central data processingcircuit) 203A-2. Similarly, sensor data processing circuit 263A-2 isconfigured to receive single-sensor data values in parallel from sensorgroup 261A-2, and finger-level sensor data signal FSD-2 is transmittedover a single elongated conductor 255A-2 to gripper controller 203A-2.In some embodiments the generation of finger-level sensor data signalsFSD-1 and FSD-2 involves converting single-sensor data values fromanalog values to digital values. In any case, finger-level sensor datasignals FSD-1 and FSD-1 form measured physical parameter values PPV (seeFIG. 2) that are provided from sensor arrays 260A-1 and 260A-2 togripper controller 203A-1. Although not shown in FIG. 3, range datagenerated by range/distance sensor 270A and image data generated bycamera(s) 280A is also provided to gripper controller 203A-2.

Referring to the right-side portion of FIG. 3, in the exemplaryembodiment various signals and power Vps are transmitted to gripper 250Aalong corresponding conductors (wires) during robotic operations. Forexample, system gripper control signals SGC and operating system updatesOSU are transmitted to gripper 250A via a first signal line 107 andtactile information TI generated by gripper controller 203A-2 istransmitted to one or more of interface computer 150A or arm controller203A-1 on a second signal line 109. Both signal transmissions passthrough a connector 257A, with system gripper control signals SGC beingsupplied to finger actuator 256A and/or gripper controller 203A-2, andoperating system updates OSU being supplied to gripper controller203A-2.

FIGS. 4 and 5 are block diagrams showing portions of a robotic system200B, which forms part of a work cell (not shown) implemented in arobotic network according to another exemplary embodiment of the presentinvention.

FIG. 4 shows portion of a robotic gripper finger 254B including amultimodal sensor array 260B in accordance with another exemplaryembodiment of the present invention. Similar to that of the previousembodiment, multimodal sensor array 260B includes a sensor group/set261B and a sensor data processing circuit 263B that are disposed on acontact region of robotic finger 254B. and configured One or more othergripper fingers, a gripper mechanism and other portions of ahierarchical sensor architecture that includes sensor group 261A and asensor data processing circuit 263A are omitted for brevity.

Referring to the left side of FIG. 4, sensor group 261A includes apressure sensor array 262B-1, one or more temperature sensors 262B-2,one or more vibration/texture sensors 262B-3, and one or more proximitysensors 262B-4. In one embodiment, each pressure sensor of pressuresensor array 262B-1 includes one of a strain gauge, a capacitivepressure sensor, a cavity-based pressure sensor, a piezoelectric sensorand a piezoresistive sensor, where each sensor is configured to generatea single-sensor data value SSD-1. Temperature sensors 262B-2 are omittedin some embodiments, and in other embodiments are implemented usingresistive temperature detectors (RTD), thermoelectric sensors or othervariants, and are configured to generate single-sensor data valuesSSD-2. Vibration/texture sensors 262B-3 are omitted in some embodiments,and in other embodiments are implemented using either piezoelectric orpiezoresistive elements or using a MEMS-based sensor configured todetect vibrations, and are configured to generate single-sensor datavalues SSD-3. Proximity sensors 262B-4 are omitted in some embodiments,and in other embodiments are implemented using acapacitive-coupling-type sensing element, and are configured to generatesingle-sensor data values SSD-4.

Sensor data processing circuit 263B includes signal conditioningcircuits 264B-1 to 264B-4, a sensor data collection circuit 265B and afinger-level sensor data generation circuit 267B. Signal conditioningcircuits 264B-1 to 264B-4 are respectively disposed to conditionsingle-sensor data values SSDA-1 to SSDA-4, and to pass the conditionssensor data values in parallel to sensor data collection circuit 265B.Sensor data collection circuit 265B receives the parallel sensor datavalues and transmits them to finger-level sensor data generation circuit267B via a serial signal line 266B. Finger-level sensor data generationcircuit 267B converts sensor data values SSD-1 to SSD-4 into afinger-level sensor data signal FSD that is transmitted on a serialsignal line 255B for processing by a gripper controller 203B-2 (shown inFIG. 5).

FIG. 5 shows a greatly simplified gripper 250B utilized in a roboticsystem 200B in the manner described above, where gripper 250B includestwo gripper fingers 254B-1 and 254B-2 are operably connected to a baseframe structure 252B and manipulated by way of actuators 256B-1 and256B-2 in a manner similar to that described above with reference toFIG. 3. Other features and details associated with the mechanism ofgripper 250B are omitted for brevity.

Similar to the fingers of gripper 250A (FIG. 3), gripper fingers 254B-1and 254B-2 respectively including associated multimodal sensor arrays260B-1 and 260B-2 (i.e., multimodal sensor arrays 260B-1 and 260B-2 arerespectively mounted on associated gripper fingers 254B-1 and 254B-2).In addition, each multimodal sensor array 260B-1 and 260B-2 isconfigured in the manner described with reference to FIG. 4 to includeboth an associated sensor set 261B-1 and 261-2 and an associated sensordata processing circuit 263B-1 and 263B-2. Specifically, multimodalsensor array 260B-1 includes sensor set 261B-1 and an associated sensordata processing circuit 263B-1, where sensor group/set 261B-1 includessensors 262B-1 to 262B-4 (shown in FIG. 4) that respectively transmitsingle-sensor sensor data (measured physical parameter) values SSD-11,SSD-12, SSD-13 and SSD-14 to sensor data processing circuit 263B-1, andsensor data processing circuit 263B-1 is configured as shown in FIG. 4to generate finger-level sensor data signal FSD-1, which is transmittedon signal line 255B-1 to a first serial data input terminal of grippercontroller 203B-2. Similarly, multimodal sensor array 260B-2 includessensor set 261B-2 and an associated sensor data processing circuit263B-2, where sensor set 261B-2 generates/transmits single-sensor sensordata values SSD-21, SSD-22, SSD-23 and SSD-24 to sensor data processingcircuit 263B-2, and sensor data processing circuit 263B-2 is configuredto generate/transmit finger-level sensor data signal FSD-2 on signalline 255B-2 to a second serial data input terminal of gripper controller203B-2. Robotic system 200B also includes a radar (range/distance)sensor 270B and cameras 280B-1 and 280B-2 that are also disposed ongripper 250B and configured such that radar sensor 270B generates rangedata RD that is transmitted via an associated signal line 255B-3 to athird input terminal of gripper controller 203B-2, and cameras 280B-1and 280B-2 generates image data ID2 that is transmitted via signalline(s) 255B-4 to a fourth input terminal of gripper controller 203B-2.

Gripper controller 203B-2 is disposed on base structure 252B of roboticgripper 250B, and is coupled to receive finger-level sensor data signalsFSD-1 and FSD-2 from sensor data processing circuits (SDPC) 263B-1 and263B-2, respectively by way of elongated conductors 255B-1 and 255B-2that respectively extend partially along fingers 254B-1 and 254B-2. Forexample, gripper controller 203B-2 is coupled to receive finger-levelsensor data signals FSD-1 from SDPC 263B-1 by way of elongated conductor255B-1, which extends partially along gripper finger 254B-1 and aportion of base structure 252B to gripper controller 203B-2. In thisembodiment gripper controller 203B-2 includes a data integration andanalysis circuit 292B that is configured to generate tactile informationTI, for example, by combining finger-level sensor data signals FSD-1 andFSD-N (i.e., physical parameter data), range data RD and image data ID2,and configured to transmit tactile information TI by way of atransceiver circuit 295B (e.g., a USB circuit as shown, or an ethernetcircuit) to one of an interface computer (IC) or a robot arm controller(RAC) in the manner described above. In addition, data integration andanalysis circuit 292B is configured to process finger-level sensor datasignals FSD-1 and FSD-N, range data RD and image data ID2, and totransmit either system gripper control signals SGC or finger controlsignals FC to an actuator control circuit 297B in accordance with adecision process similar to that described above, whereby system grippercontrol signals SGC are transmitted to finger actuators 256B-1 and256B-2 during normal operating conditions, and finger control signals FCare transmitted to finger actuators 256B-1 and 256B-2 during abnormaloperating conditions.

Although the present invention has been described with respect tocertain specific embodiments, it will be clear to those skilled in theart that the inventive features of the present invention are applicableto other embodiments as well, all of which are intended to fall withinthe scope of the present invention. For example, although the roboticsystem of each work cell is described above as comprising a single robotmechanism, each work cell may include two or more robot mechanismswithout departing from the spirit and scope of the present invention.Similarly, although each robot mechanism is described herein asincluding a single end-effector, each robot mechanism may include two ormore end-effectors/grippers without departing from the spirit and scopeof the present invention. Moreover, the robotic system configurationsdescribed herein may be modified to include one or more featuresassociated with the flex-rigid sensor array structures described inco-owned and co-filed U.S. patent application Ser. No. 16/832,755entitled “FLEX-RIGID SENSOR ARRAY STRUCTURE FOR ROBOTIC SYSTEMS”, or thetactile perception structures described in co-owned and co-filed U.S.patent application Ser. No. 16/832,690 entitled “TACTILE PERCEPTIONAPPARATUS FOR ROBOTIC SYSTEMS”, or the robotic gripper described inco-owned and co-filed U.S. patent application Ser. No. 16/832,800entitled “ROBOTIC GRIPPER WITH INTEGRATED TACTILE SENSOR ARRAYS”, all ofwhich being incorporated herein by reference in its entirety.

The invention claimed is:
 1. A network comprising a plurality of workcells that are respectively operably configured to communicate with acloud server, wherein each work cell of the plurality of work cellsincludes an interface computer and an associated robotic systemincluding a control circuit, where said interface computer of each saidwork cell is coupled between the cloud server and the control circuit ofthe associated robotic system, wherein said robotic system of each workcell further includes a robot mechanism including a plurality of robotstructures and a plurality of first actuators configured to controlrelative movement of said robot structures in response to controlsignals received from the control circuit of the associated roboticsystem, said each robot mechanism also including an end effectoroperably connected to a distal end of said plurality of robot structuresand including at least one second actuator, wherein said end effectorfurther includes at least one associated multimodal sensor arraycomprising a plurality of sensors respectively configured to measure aplurality of physical parameter values during a robotic operationinvolving interactions between said end effector and a target object,wherein the control circuit of each said work cell is configured tocontrol said robot mechanism during said robot operation by generatingcontrol signals in accordance with an operating system, and bytransmitting said control signals to at least one of said first andsecond actuators, and wherein the cloud server is configured to receiveat least some of said measured physical parameter values from saidplurality of work cells, and configured to automatically transmit anupdated operating system to the control circuit of each of saidplurality of work cells, whereby said each control circuit generatessaid control signals in accordance with an initial operating systemprior to said automatic update, and said each control circuit generatessaid control signals in accordance with said updated operating systemafter said automatic update.
 2. The network of claim 1, wherein theupdated operating system differs from the initial operating system inthat the updated operating system includes a stored set of indicatorparameter values and an associated secondary robot control signal, andwherein, after the cloud server automatically transmits said updatedoperating system, the control circuit of each said work cell iscontrolled by the updated operating system to transmit primary robotcontrol signals to one or more actuators of said plurality of actuatorsof the associated robot mechanism while currently measured physicalparameter values received from said at least one associated multimodalsensor array fails to match said a stored set of indicator parametervalues, and is controlled to transmit said associated secondary robotcontrol signals to said one or more actuators when the currentlymeasured physical parameter values match said associated set of storedindicator parameter values.
 3. The network of claim 1, wherein the robotmechanism of each said work cell comprises a robot arm having a fixedbase disposed at one end and a robotic gripper connected to said distalend and having a plurality of gripper fingers, and wherein the controlcircuit of each said work cell includes a robot arm controller and agripper controller, said robot arm controller being configured tocontrol the robot arm by way of transmitting arm control signals to oneor more arm actuators operably disposed on the robot arm, and saidgripper controller being configured to control operations performed bythe robotic gripper by way of transmitting finger control signals to oneor more finger actuators that are operably disposed on the roboticgripper.
 4. The network of claim 3, wherein each said multimodal sensorarray is disposed on a corresponding said gripper finger and includes aplurality of sensors that are respectively operably configured togenerate an associated set of said measured physical parameter values inresponse to interactions between said corresponding gripper finger andthe target object during each said interaction between said end effectorand said target object, and wherein each said gripper controller isfurther configured to generate both the finger control signals andtactile information in accordance with the measured physical parametervalues received from one or more of said multimodal sensor arraysdisposed on its associated robotic gripper, and said gripper controllerfurther includes a transceiver circuit configured to transmit thetactile information to one of said arm control circuit and saidinterface computer.
 5. The network of claim 4, wherein the robot armcontroller is further configured to transmit system gripper controlsignals to the finger actuators while the measured physical parametervalues fail to match at least one stored set of indicator parametervalues, and the gripper controller is further configured to transmit thefinger control signals to the finger actuators when the measuredphysical parameter values match said associated set of stored indicatorparameter values.
 6. The network of claim 1, wherein the robotic systemof each said work cell includes at least one of a range sensor and acamera, and wherein at least one of the arm controller, the grippercontroller and the interface computer is configured to receive data fromsaid at least one of said range sensor and said camera.
 7. The networkof claim 1, wherein each said robotic gripper includes a range sensorand one or more cameras, and said gripper controller is configured toreceive range data from the range sensor and image data from the one ormore cameras.
 8. The network of claim 1, wherein each said multimodalsensor array is disposed on a corresponding gripper finger, wherein eachsensor of said plurality of sensors of said each multimodal sensor arrayis configured to generate an associated measured physical parametervalue, and wherein said plurality of sensors of said each multimodalsensor array comprises an array of pressure sensors and one or more of atemperature sensor, a vibration sensor, and a proximity sensor.
 9. Thenetwork of claim 8, wherein each pressure sensor of said array ofpressure sensors comprises of one of a strain gauge, a capacitivepressure sensor, a cavity-based pressure sensor, a piezoelectric sensorand a piezoresistive sensor.
 10. The network of claim 8, wherein eachsaid multimodal sensor array further comprises a sensor data processingcircuits disposed on said corresponding gripper finger and configured togenerate a corresponding finger-level sensor data signal in response tosaid measured physical parameter value generated by said plurality ofsensors, and wherein said sensor data processing circuits is configuredto transmit said corresponding finger-level sensor data signal to saidgripper control circuit.
 11. The network of claim 8, wherein the controlcircuit of each said work cell is configured to control said robotmechanism during said robot operation using one of an artificialintelligence model and a machine-learning model.